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Introduction 


Based  on  a  request  from  Dr.  John  Carpenter,  COL,  MC,  Chief,  Department  of 
Medicine,  Brooke  Army  Medical  Center,  a  statistical  consultation  was  performed 
to  determine  significant  variables  affecting  the  rate  of  surgeons'  compliance 
with  recommendations  resulting  from  consultations  with  internists.  The  request 
for  analytical  support  was  based  on  a  six  month  study  of  compliance  which  pro¬ 
duced  data  on  419  consultations,  resulting  in  1705  recommendations .  The  study 
was  performed  by  Or.  Larry  Pupa,  CPT,  MC,  Fellow,  Cardiology  Service  and  in¬ 
volved  recommendations  by  several  residents.  One  purpose  of  the  study  was  to 
evaluate  the  effects  of  implementing  results  from  a  major  study  of  compliance 
factors  reported  by  Sears  and  Charlson  (1983). 

Raw  data  was  provided  to  HCSCIA  in  hard  copy  form.  The  data  was  edited  by 
HCSCIA  personnel  as  it  was  entered  into  the  computer  for  analysis  and  many 
coding  inconsistencies  were  discovered.  These  problems  were  resolved  with 
Department  of  Medicine  personnel  resulting  in  419  "good  consults"  upon  which  to 
base  the  analysis.  All  statistical  analysis  was  performed  using  the  Statistical 
Package  for  the  Social  Sciences  (SPSS).  A  summary  of  the  important  statistical 


results  is  provided  in  the  remainder  of  this  report. 


Variables  Analyzed 


!  O'M!  IT 


CONSULT  SPECIFIC: 


AGE  -  Age  of  Patient  (1-99) 

SEX  -  Sex  of  Patient  (M  or  F) 

TIME  -  Time  of  Consult  (Preop,  Postop,  or  Postop  >  24  hours) 

CLINDEX  -  Clinical  Index  of  Patient  based  on  McCabe  or  Anes  (A[I,II], 
BCIII],  or  CCIV.V]) 

SPDX  -  Sub-specialty  Area  of  Diagnosis  (10  sub-specialties) 

NREC  -  Number  of  Recommendations  (Size  of  List  1-8) 

CONT  -  Contact  with  Surgeon  (Immediate,  After  24  hours,  or  None) 


wailabitity  Codes 


Avail  and/or 
Special 


RECOMMENDATION  SPECIFIC: 


CROC  -  Importance  (Crucial  or  Non-Crucial) 

FOIL  -  Compliance  (Followed  or  Not  Followed) 

TREC  -  Type  of  Recommendation  (Diagnostic  or  Therapeutic) 

PROC  -  Procedure  (None,  Procedure  by  MD,  or  Procedure  by  RN) 

DRTP  -  Drug  Type  (None  or  7  types  of  drugs) 

DRMN  -  Drug  Manipulation  (None,  Start,  Stop,  Continue,  or  Adjust) 

Statistical  Analysis 

Differences  between  proportions  were  assessed  using  the  chi-square  test  of 
independence  when  sample  size  was  sufficient  and  with  Fisher's  Exact  Test  when 
sample  size  was  small.  Differences  between  means  of  continuous  variables  were 
evaluated  using  two-tailed  t  tests  for  two  groups  and  analysis  of  variance  F 
ratios  for  more  than  two  groups.  Two-group  discriminant  analyses  were  performed 
to  evaluate  the  relative  importance  of  various  consult  and  recommendation  char¬ 
acteristics  for  predicting  compliance.  In  most  cases,  a  0.02  level  of  signifi¬ 
cance  was  required  as  the  judge  of  statistical  significance  of  results  to  reduce 
the  impact  of  multiple  tests  on  overall  statistical  error  rates.  Exceptions  are 
noted  and  exact  p-values  are  reported.  It  should  be  carefully  noted  that  in 
the  analysis  of  individual  recommendations  (n=1705),  the  assumption  of  indepen¬ 
dent  observations  required  by  the  chi-square  test  was  violated  in  that  multiple 
recommendations  on  an  individual  patient  cannot  realistically  be  considered  as 
independent  trials.  This  may  result  in  an  artificially  inflated  sample  size  for 
some  chi-square  tests  and,  potentially,  artificially  inflated  significance 
levels  (smaller  p  values). 

Results  of  Analysis 

OVERALL: 

Tables  1  and  2  provide  an  overall  descriptive  analysis  of  variables  which 
turned  out  to  be  "worth"  analyzing.  Characteristics  of  the  419  consultations 


Recommendations  Recommendations  Followed 

Variable  Consults  Per  Consult  Crucial  Non  Crucial  Total 
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CHARACTERISTICS  OF  419  CONSULTATIONS 


ADDITIONAL  CHARACTERISTICS  OF  1705  RECOMMENDATIONS 


and  the  resulting  1705  recommendations  are  displayed.  Totals  are  provided  in 
each  section  of  the  tables  to  indicate  the  presence  of  missing  observations. 
Compliance  is  shown  separately  for  Crucial  and  Non-Crucial  recommendations  based 
on  importance  to  subsequent  analysis.  Results  of  specific  statistical  com¬ 
parisons  and  additional  analyses  are  given  below. 

CONSULT  SPECIFIC  (N  =  419): 

Tables  3  and  4  display  the  distribution  of  the  overall  number  of  recommen¬ 
dations  and  number  of  crucial  recommendations  across  the  419  consults.  It  is 
noteworthy  that  more  than  75%  of  the  lists  were  no  longer  than  five  recommen¬ 
dations  and  less  than  10%  of  the  lists  contained  greater  than  two  crucial  recom¬ 
mendations  . 

Interesting  relationships  between  the  size  of  consult  lists  and  the  Type  of 
Consult,  Clinical  Index  of  the  patient,  and  Contact  with  Surgeons  are  shown  in 
Table  5.  Preop  Clearance  consults  tend  to  result  in  significantly  fewer  recom¬ 
mendations  than  Specific  Management  Problems  or  General  consults  (p  =  0.0128); 
McCabe  B  and  C  patients  require  significantly  more  recommendations  than  McCabe  A 
patients  (p  =  0.0001);  while  there  is  a  tendency  for  immediate  contact  to 
increase  with  the  length  of  the  list.  This  last  relationship  is  not  statisti¬ 
cally  significant  (p  *  0.0682)  but  is  certainly  intuitive.  These  relationships 
are  maintained  even  when  the  Size  of  List  is  not  collapsed  into  two  categories 
and  appear  to  be  quite  linear  with  the  number  of  recommendations. 

Tables  6  and  7  display  the  distributions  of  overall  per  patient  compliance 
with  recommendations  from  the  419  consults  and  compliance  with  crucial  recom¬ 
mendations  for  the  101  patients  who  had  crucial  recommendations,  respectively. 

It  is  noteworthy  that  in  fewer  than  10%  of  the  cases  shown  in  Table  7  were  less 
than  100%  of  the  crucial  recommendations  followed. 


Table  3 


FREQUENCY  DISTRIBUTION  OF  ALL  RECOMMENDATIONS 


NREC 

Freq 

Percent 

Cumm  Percent 

1 

38 

9.1% 

9.1% 

2 

62 

14.8% 

23.9% 

3 

74 

17.7% 

41.5% 

4 

86 

20.5% 

62.1% 

5 

64 

15.5% 

77.3% 

6 

40 

9.5% 

86.9% 

7 

21 

5.0% 

91.9% 

8 

34 

8.1% 

100.0% 

419 

100.0% 

Table  4 


FREQUENCY  DISTRIBUTION  OF  CRUCIAL  RECOMMENDATIONS 


NCREC 

Freq 

Percent 

Cumm  Percent 

0 

318 

75.9% 

75.9% 

1 

45 

10.7% 

86.6% 

2 

20 

4.8% 

91.4% 

3 

13 

3.1% 

94.5% 

4 

12 

2.9% 

97.4% 

5 

7 

1.7% 

99.1% 

6 

3 

0.7% 

99.8% 

7 

1 

0.2% 

100.0% 

8 

0 

0.0% 

100.0% 

419 

100.0% 

RELATIONSHIPS  WITH  SIZE  OF  LIST 


Table  7 


PER  PATIENT  CRUCIAL  RECOMMENDATION  COMPLIANCE  RATE 


Compliance  Rate 

Freq 

Percent 

Cumm  Percent 

0  -  49% 

4 

3.96% 

3.96% 

50  -  74% 

1 

0.99% 

4.95% 

75  -  99% 

4 

3.96% 

8.91% 

100% 

92 

91.09% 

100.00% 

101 

100.00% 

There  was  a  weak  but  statistically  significant  correlation  between  per 
patient  compliance  rate  and  number  of  recommendations  (n  =  419,  r  =  -0.1324,  p  ^ 
0.003).  This  relationship  is  further  illustrated  in  Figure  1  which  shows  list 
size  from  1  to  8  and  compliance  rate  categorized  as  follows:  0-74%,  75-99%,  and 
100%.  Percent  of  consults  falling  into  each  category  is  displayed  on  the  ver- 
ticle  axis.  A  chi-square  analysis  of  these  frequencies  revealed  a  highly  signi 
ficant  relationship  (chi-square  =  90.4,  d.f.  =  14,  p  <  0.0001).  In  fact,  a 
simple  correlation  between  number  of  recommendations  and  percent  of  consults 
with  100%  compliance  produced  r  =  -0.9322  (n  =  8,  p  <  0.001). 

Other  interesting  per  consult  results  include  the  following.  The  General 
type  consults  showed  a  lower  overall  compliance  rate  than  other  types  combined. 
(Fisher's  Exact  Test,  p  =  0.018)  A  two  group  discriminant  analysis  (100%  com¬ 
pliance  versus  less  than  100%  compliance)  demonstrated  the  statistically  signif 
icant  ability  of  Age  (<60  vs.  >60),  Type  of  Consult  (General  vs.  Other), 
Clinical  Index  of  Patient  (6  vs.  Other),  and  Size  of  List  (1-5  vs.  >5)  to 
predict  compliance.  However,  only  61%  of  the  419  consults  were  correctly  clas¬ 
sified  using  the  model  with  these  variables.  One  would  do  better  simply  pre¬ 
dicting  100%  compliance  in  all  cases  with  an  overall  successful  classification 

9 


NUMBER  OF  RECOMMENDATIONS 
ZZl  0-74%  ES  75-99%  Ea  ^00% 


rate  of  73X.  No  other  statistically  significant  relationships  were  found  in  the 
per  consult  analysis. 

RECOMMENDATION  SPECIFIC  (N  =  1705): 

Analysis  of  all  appropriate  variables  and  their  effect  on  compliance  with 
individual  recommendations  was  conducted.  Most  of  the  applicable  descriptive 
statistics  were  displayed  in  Tables  1  and  2  and  will  not  be  repeated  here.  Once 
again  it  should  be  noted  that  the  1705  recommendations  are  not  Independent 
observations  and  the  statistical  results  should  be  interpreted  with  caution. 


However,  most  of  the  results  are  intuitive  and  supported  by  previous  research 
and  many  are  highly  significant. 

In  the  analysis  of  overall  compliance  with  the  1705  individual  recommen¬ 
dations,  the  variables  Age,  Sex,  Time  of  Consult,  Type  of  Consult,  Clinical 
Index,  Subspecialty  Diagnosis,  Number  of  Recommendations,  Procedures,  and 
Contact  with  Surgeon  were  not  significantly  related  to  compliance.  Whether  or 
not  a  recommendation  was  denoted  as  Crucial  had  a  significant  impact  on 
compliance  (96.1%  for  Crucial  versus  89.4%  for  Non-Crucial,  p  =  0.0014),  There 
was  also  a  significant  difference  between  compliance  rates  for  Diagnostic  versus 
Therapeutic  recommendations  (87.4%  versus  94.1%,  p  =  0.0001).  Recoiranendations 
involving  Drugs  were  followed  a  significantly  higher  percentage  of  the  time 
(93.6%  versus  89.2%  for  Non-Drug  recommendations,  p  =  0.008).  In  addition,  for 
those  recommendations  involving  drug  therapy,  it  made  a  "close  to  "  significant 
difference  as  to  what  action  was  called  for.  Compliance  with  "Continue"  recom¬ 
mendations  was  100%  whereas  Start,  Stop,  or  Adjust  recommendations  were  followed 
about  92%  of  the  time  (p  =  0.0479). 

Since  the  impact  of  specifying  that  a  recommendation  was  Crucial  appeared  so 
significant,  it  was  decided  to  analyze  this  variable  in  two  different  ways. 
First,  an  analysis  of  the  impact  of  all  other  variables  on  compliance  was 
conducted  separately  for  Crucial  versus  Non-Crucial  recommendations.  These 
results  may  be  summarized  as  in  Table  8. 

The  significant  relationships  indicated  include  the  following.  Type  of 
Consult:  One-hundred  percent  of  the  Crucial  recommendations  for  Specific 
Management  Problems  were  complied  with  compared  to  93.7%  compliance  with  other 
types  of  consults  (p  =  0.008).  This  indicates  a  greater  tendency  for  surgeons 
to  follow  important  recommendations  concerning  problems  they  themselves  have 


Table  8 


IMPACT  OF  VARIABLES  ON  COMPLANCE  RATE 
CONTROLLING  FOR  CRUCIAL  VERSUS  NON-CRUCIAL 


Importance  of  Recommendation 

VARIABLE 

NON-CRUCIAL 

CRUCIAL 

Age 

N.S. 

N.S. 

Sex 

N.S. 

N.S. 

Time  of  Consult 

N.S. 

N.S. 

Type  of  Consult 

N.S. 

S. 

Clinical  Index 

N.S. 

N.S. 

Sub-Specialty  Dx 

N.S. 

N.S. 

Size  of  List 

N.S. 

N.S. 

Contact  w/Surgeon 

N.S. 

N.S. 

Type  Recommendation 

S. 

N.S. 

Procedures 

N.S. 

S. 

Drug  (versus  None) 

S. 

N.S. 

S.  =  Significant 
N.S.  =  Not  Significant 


identified.  Type  of  Recommendation:  Whether  a  recommendation  was  Diagnostic  or 
Therapeutic  made  a  significant  difference  in  compliance  with  Non-Crucial  recom¬ 
mendations  (86.2%  versus  93.6%,  p  =  0.0001)  but  this  difference  was  not  signifi¬ 
cant  for  Crucial  recommendations  (95.3%  versus  97.1%).  This  may  mean  that 
Crucial  or  not  is  more  important  than  the  nature  of  the  recommendation. 
Procedures:  When  a  procedure  was  suggested,  only  88.5%  of  the  Crucial  recommen¬ 
dations  were  followed,  whereas  Crucial  recommendations  not  involving  procedures 
were  complied  with  98.8%  of  the  time  (p  =  0.0003,  but  there  were  only  61  of  the 
1705  recommendations  which  were  for  crucial  procedures).  Drug:  Finally,  for 
Non-Crucial  recommendations,  whether  or  not  a  drug  was  prescribed  significantly 
impacted  on  compliance  (92.7%  versus  88.4%  for  no  drug,  p  =  0.0235).  This 
variable  had  no  significant  effect  on  compliance  with  Crucial  recommendations. 


To  further  explore  the  importance  of  indicating  that  a  recommendation  was 
Crucial,  an  attempt  was  made  to  separately  examine  this  impact  within  categories 
of  the  other  variables.  These  results  are  summarized  in  Table  9.  P  values  are 
shown  in  all  cases  of  statistical  significance.  Crucial  versus  Non-Crucial  made 
a  significant  difference  in  compliance  on  both  lists  of  five  or  less  and  greater 
than  five  recommendations.  Crucial  made  a  difference  within  Diagnostic  but  not 
Therapeutic  recommendations.  This  significance  was  repeated  within  recommenda¬ 
tions  involving  Immediate  Contact  with  Surgeons  but  it  is  not  clear  which  is  the 
dependent  variable  in  this  instance.  Crucial  recommendations  for  Specific 
Management  Problems  were  followed  a  significantly  greater  percentage  of  the  time 
than  Non-Crucial  recommendations  for  similar  situations.  This  is  important  in 
that  even  in  these  cases  where  surgeons  have  made  specific  requests,  indicating 
that  the  recommendation  is  Crucial  still  makes  a  significant  difference.  Final¬ 
ly,  Crucial  made  a  significant  difference  when  No  Drug  was  involved  in  the 
recommendation  but  not  in  the  case  of  Drug  involvement.  This  is,  of  course,  due 
to  the  overall  high  compliance  rate  for  Drug  recommendations  in  general. 

The  final  statistical  analysis  was  a  two  group  (followed  or  not)  discrimi¬ 
nant  analysis  of  the  variables  effecting  compliance  with  the  1705  individual 
recommendations.  The  variables  Crucial,  Number  of  Recommendations,  and  Type  of 
Recommendation  were  significant  in  this  analysis  but,  once  again,  the  classifi¬ 
cation  results  of  this  model  were  poor  with  only  57%  of  the  overall  recommenda¬ 
tions  being  correctly  assigned. 


IMPACT  OF  CRUCIAL  RECOMMENDATIONS  ON  COMPLIANCE  RATE 
CONTROLLING  FOR  OTHER  VARIABLES 
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